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ABSTRACT 

According to the education and training requirements 
of the Family Support Act of 1988, the Connecticut Department of 
Income Maintenance (DIM) is required to approve criteria or standards 
for determining the satisfactory progress of Aid to Families with 
Dependent Children (AFDC)/Job Connection clients enrolled in 
education and skill training programs. This report recommends draft 
standards for determining clients' satisfactory progress in basic 
skills, classes in English as a Second Language, high school 
completion, higher education, and skill training programs; and it 
provides guidelines for selecting high quality programs. Following a 
brief introductory chapter, chapter 2 describes current practices in 
determining satisfactory progress. Chapter 3 proposes the standards 
and also provides guidelines for selecting quality programs, 
including how to determine the probationary period during which 
clients who have not met the standards may be given another 
opportunity and how to determine whether mitigating circumstances 
were responsible for a client failing to meet the standards. Chapter 
A addresses factors that affect implementation of the standards 
(interagency cooperation, staff training, and provision of 
information about service providers to whom case managers refer 
clients). Appendices contain the questionnaire used to survey 
Connecticut's DIM staff prior to the preparation of this report and a 
list of companies, institutions, and agencies that provider services 
to Job Connection clients. (CML) 
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PREFACE 



The Family Support Act of 1988 calls for State IV-A agencies to approve standards 
for determining the satisfactory performance of Aid to Families with Dependent Children 
clients in education and skill training programs. In response to this requirement, the 
Connecticut Department of Income Maintenance (DIM)/Job Connection, through an 
interagency agreement with the Connecticut Department of Education, commissioned 
COSMOS Corporation to conduct a project to develop draft standards for determining 
Job Connection clients' satisfactory progress in basic skills, English as a second 
language, high school completion, higher education, and skill training programs. In 
addition to developing draft standards, COSMOS was to specify draft guidelines for 
determining the quality of education and skill training programs that provide services in 
these areas. 

This document is the final report for this project. Described in the report are the 
results of a review of existing information and the collection of new data concerning 
current practices undertaken by DIM district Job Connection staff to determine clients' 
satisfactory progress and select quality programs. Included in this discussion is 
information concerning other states' procedures for assessing clients' progress in 
education programs. Also presented in the report are options and recommendations for 
draft standards for determining satisfactory progress and guidelines for selecting quality 
programs, as well as issues that need to be considered in implementing these standards. 

This report is submitted to the Connecticut Department of Income Maintenance and 
the Connecticut Department of Education, Bureau of Adult Education and Training. The 
project was funded through an interagency agreement between the Connecticut Depart- 
ment of Income Maintenance and the Connecticut Department of Education, under a 
Purchase Order from the Capitol Region Education Council to COSMOS Corporation. 
The COSMOS Corporation staff who conducted this project were: Judith A. Alamprese, 
the project's director and author of this report; Julie Beaver, a research assistant who 
participated in data collection and analysis activities and prepared the tables included in 
the report; and Michael Caruso and Timothy Duggan, research assistants who partic- 
ipated in the project's data collection and analysis activities. 
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EXECUTIVE SUMMARY 

Introduction to the Project 

According to the education and training requirements of the Family Support Act of 
1988, the Connecticut Department of Income Maintenance (DIM) is required to approve 
criteria or standards for determining the satisfactory progress of Aid to Families with 
Dependent Children (AFDC)/Job Connection clients enrolled in education and skill 
training programs. The legislation states that these standards of progress must be 
measured on the basis of less than a year, include a qualitative measure of the 
participant's progress (such as a grade point average), and include a quantitative measure 
of the participant's progress (such as a time limit for the completion of the program or 
course of study). The standards also may include consideration for a probationary 
period and provision for mitigating circumstances. 

State welfare agencies were given a three-year period in which to develop and 
implement the standards. As an initial step in developing standards, the DIM Job 
Connection staff have used the standards of progress that were developed by service 
providers and those that were negotiated with DIM-funded service providers. This 
experience has provided the background information for DIM to use in specifying final 
standards for determining Job Connection clients' satisfactory progress in education and 
skill training programs. 

Since January 1990, the Job Connection staff in DIM have worked with the 
Connecticut Department of Education staff and others to collect information about the 
types of practices undertaken by education and skill training providers to assess clients' 
progress in programs as a step in determining standards of progress. Issues regarding 
the types of reasonable standards that can be set for determining progress and methods 
for transferring information between education and skill training providers and DIM 
district Job Connection staff have been the subjects of discussion in State and local 
program meetings. In addition, initiatives such as the Connecticut Adult Performance 
Program (CAPP), PROJECT LIFE (Learning Incentives for Employment), and CETO 
have provided information and resources regarding client assessment and interagency 
transfer of client data. 
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An array of information concerning Job Connection participants' satisfactory 
progress in education and skill training programs has emerged from the numerous State 
and local discussions and from the experiences of the implementation of the varied 
initiatives. Because of the breadth of information, the DIM Job Connection adminis- 
trative staff thought that it would be beneficial to have an independent consultant review 
this information and prepare draft standards of satisfactory progress. DIM, through an 
interagency agreement with the Connecticut Department of Education, commissioned 
COSMOS Corporation to carry out a project during the period January-July 1992. This 
report presents the findings from the project, which had the following objectives: 

■ Review existing information concerning the 
development of standards for satisfactory progress; 

■ Collect information from DIM district case managers 
concerning their procedures for determining satisfactory 
progress, their methods for collecting this information 
from service providers, and their methods for deter- 
mining quality programs; 

■ Collect information from a small sample of States 
regarding their policies for determining satisfactory 
progress as specified in the Family Support Act of 
1988; 

■ Develop draft standards for determining satisfactory 
progress in basic skills, English as a second language 
(ESL), high school completion, higher education, and 
skill training programs; 

■ Develop draft guidelines for determining quality 
programs; and 



■ Identify issues that should be considered in the 

implementation of standards, particularly in terms of the 
types of staff training that might be provided. 



Data Collection Methods and Data Sources 

A variety of existing sources of information was reviewed and new data were 
collected to understand the practices that district Job Connection staff currently use to 
determine the extent to which clients' progress in programs is satisfactory and to select 
quality programs. The following information sources were examined as part of the 
project's activities: 

■ Notes from Regional Interagency Steering Committee 
(RISC) meetings held during 19904991; 

■ Reports written about PROJECT LIFE; 

■ Memoranda issued by Job Connection and Department 
of Education staff; 

■ Policy documents issued by DIM and other State and 
local welfare agencies; 

■ Sample data collection forms submitted by local Job 
Connection and local adult education program staff; and 

■ Data concerning CAPP assessments results during 1987- 
1990. 

In addition to analyzing existing information, the project's staff collected new 
information through the following four methods: 1) the conduct of three focus groups 
with local Job Connection and adult education staff, and representatives from higher 
education and skill training agencies; 2) the conduct of a meeting with the project's State 
Advisory Committee; 3) the conduct of a mail survey of Job Connection case managers 
in the six DIM districts, which had a response rate of 52 percent; and 4) the conduct of 
telephone interviews with representatives from four State welfare agency Job 
Opportunities and Basic Skills (JOBS) staff. 

Findings Regarding Current Practices in Determining Clients' Satisfactory Progress 

As background to preparing draft standards of satisfactory progress and guidelines 
for selecting quality programs, the project staff identified the types of information that 
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Job Connection staff currently collect about clients* progress, the criteria they use to 
determine whether the progress is satisfactory, the methods they use to obtain this 
information from education and skill training service providers, and the factors that have 
facilitated and inhibited the transfer of information between local agencies. Also 
identified were the criteria that Job Connection staff use to determine the quality of 
service providers to whom they refer clients and the barriers that these staff have 
encountered in obtaining information to make these judgements. In addition to these 
data, die project staff collected information from five State welfare agencies concerning 
their policies for determining satisfactory progress in education programs. 

Types of I nformation . The results of the survey of Job Conneaion case managers 
revealed that they use the following categories of information to determine the 
satisfactory progress of their Job Connection clients: 

■ Measures of skill gain; 

■ Other education measures, such as grade or grade point 
average , teachers' written and oral assessments, and 
program certificate; 

■ Attendance; 

■ Job skill attainment progress; and 

■ Measures of clients' behavior and attitude. 

The most frequently-used measure of skill gain reported by case managers is pass rate 
on the General Educational Development (GED), followed by CAPP score. Few survey 
respondents use information from the Test of Adult Basic Education (13 percent) or the 
Adult Basic Learning Examination (16 percent). In addition to skill measures, case 
managers consider other types of educational data in determining clients' progress. All 
survey respondents reported using a grade or grade point average (G.P.A.) and most (89 
percent) said that a program certificate is evidence of progress. In terms of teachers' 
professional judgement, the majority of case managers (80 percent) indicated that they 
collect teachers' written assessment of clients, while 60 percent noted that they rely on 



vu 

teachers' verbal assessment. Another category of information that all survey respon- 
dents reported using as an indicator of client progress was attendance. 

In addition to collecting data about education measures and attendance, approxi- 
mately one third of the case managers also obtain information concerning clients' 
progress in skill training programs. The final type of information gathered by case 
managers concerns clients' change in behavior and attitude in either education or job 
training programs. Approximately half of the survey respondents indicated that they 
consider change in a client's attitude as one indicator of progress, while ono third noted 
that they accept data concerning clients' change in behavior. 

Criteria fo r Determining Prog ress. In addition to collecting data in the survey of 
case managers about the types of information that case managers use to determine 
clients' progress, the project staff also examined the criteria that case managers use to 
determine if clients' progress is satisfactory. Case managers generally consider an 
increase in test score, class grade, or grade point average as an indicator of progress. 
While an increase in score or grade is the criterion used by case managers, there is little 
data available about the amount of increase that is considered by case managers to be 
satisfactory. The information that is available indicates that there is great variation in 
the score levels, grades, and attendance rates used by service providers to determine 
client progress. This variation reflects the current practice in DIM districts of having 
local service providers set their own standards for satisfactory progress. 

For measures other than tests or grades, case managers have used the attainment of 
a certificate or the completion of a program's requirements as indicators of progress. 
Case managers also look to the institution or service provider to set a standard and asses 
their clients using this information. There is also variation in the criterion used by case 
managers regarding attendance. While the Family Support Act's regulations specify that 
JOBS participants must have an average of 75 percent attendance during a month's time, 
case managers report using rates from 70 to 80 percent in making their assessment. 

In measuring clients' progress in skill training programs, case managers generally 
consider the receipt of a certificate, or movement to the next level of training as 
evidence. They also rely on service providers to specify a standard for assessing client 
progress. 
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The final area examined in the survey of cast managers was the measurement of 
change in clients' behavior and attitude. Case managers rely on their own observations 
and those of service providers in making these assessments, rather than the collection of 
formal data through instruments or other systematic processes. Examples of the types of 
behavioral change that case managers consider are clients' ability to plan and to initiate 
contact with service providers. In the area of attitudinal change, case managers 
frequently use the criterion of clients' demonstration of a positive attitade toward their 
training program. 

The majority of case managers who responded to the survey use multiple measures 
to assess their clients' progress. Almost half reported using at least three measures of 
skill gain, and 56 percent indicated that they use four "other education measures." This 
information about the amount of data that case managers use in mfiking their judgements 
about clients is a factor in determining whether standards of satisfactory progress should 
include a variety of measures from which case managers can choose. 

Interagency Exchange of Information . The information collected from the survey of 
case managers, focus group meetings, and the review of RISC meeting reports indicates 
that Job Connection case managers and staff from service provider agencies are working 
together in a variety of ways to transfer client information across agencies. These 
methods include the use of referral forms that contain information about a client's 
demographic characteristics, educational and work history, and performance on appraisal 
instruments, and the use of forms that contain data on clients' attendance and progress in 
basic skills, ESL, secondary education, or skill training programs. In addition to the 
transfer of information in written form, case managers and service provider staff have 
established methods of communicating about clients through telephone discussions and 
meetings. These methods are particularly effective when a case manager has a question 
about the progress of a client and is able to discuss the issue with the service provider. 

Case managers generally rely on service providers to gather and transfer information 
about a client, but also receive information from clients aoout their progress. This 
practice is necessary in higher education programs, where attendance is not kept in a 
central office and the client is relied upon to provide the information to the case 
manager. 

® 10 
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While case managers have been creative in working with staff from education and 
skill training agencies in sharing information, they also have experienced some barriers 
in gathering data they need in an appropriate form and timeframe. Of particular concern 
have been case managers' difficulties in interpreting information about clients' perfor- 
mance on basic skills measures, and the frequency with which attendance information is 
transferred between agencies. Also problematic has been the unavailability of infor- 
mation from skill training agencies regarding clients' progress. 

One factor that has facilitated communication between case managers arid other 
agency staff is their personal relationships. The commitment of staff to work together to 
assist clients has mediated some of the structural difficulties staff have encountered in 
collecting the required information. 

Criteria for Determining Ouality Programs . In the survey of case managers, 
respondents identified the following four factors as being the most important in their 
selection of service providers: 1) type of skill training offered; 2) relationship between 
the skill training offered and the job market; 3) geographical location of the service 
provider; and 4) agency's rate of job placement. Also rated highly were case managers' 
personal experience with the service provider and the cost of the program. Survey 
respondents also identified the types of information that they would like to have available 
but have not been able to access. These responses included data about agencies' job 
placement rates and information about program services and requirements. 

States' Criteria for Detennining Satisfactory Progress . Part of the data collection for 
this project involved gathering information about standards for satisfaaory progress from 
other States. Five States that were early implementers of the Federal JOBS Program- 
California, Michigan, New Jersey, Massachusetts, and Pennsylvania— were selected for 
study. All of the States except for Pennsylvania provided specific information about 
determining clients' progress in programs. Pennsylvania's policy document addressed 
criteria for placing clients in programs, but did not specify the measures of progress. 
California has the most clearly delineated standards, with the required scale scores on 
the Comprehensive Adult Student Assessment System (CASAS) Survey Achievement 
being stated. Michigan uses competency tests that are administered by local providers 



X 

and relies on these providers to set the standard. New Jersey does not specify the type 
of measure that should be used, but relies on individual service providers to judge 
clients' progress. Finally, Massachusetts specifies the instrument and level for clients in 
ESL programs, and relies on local providers' use of competency tests for assessing 
clients' progress. 

Proposed Standards for Determining Clients' Satisfactory Progress and Guidelines for 
Selecting Quality Programs 

Framework for Standards and Guidelines . The regulations for the Family Support 
Act of 1988 specify broad criteria for developing standards of progress, which were 
used by the project team as one guideline in preparing the draft standards. The team 
also identified three other factors that should be considered in setting standards of 
satisfactory progress and quality program guidelines. These were: 1) the extent to 
which the standards and guidelines are reasonable and fair-i.e., are based on empirical 
data or other information about clients' capacities to perform in a specific timeframe; 2) 
the feasibility of the standards and guidelines, -i.e., the extent to which information 
needed to implement a standard is available and accessible to Job Connection case 
managers; and 3) the extent to which the standards and guidelines are flexible so that 
they can be modified to meet the special situations or needs of clients. 

Draft Standards for Satisfactory Performance . The types of information that Job 
Connection case managers use to judge their clients' progress in education and skill 
training programs were one source of data examined in developing standards. Also 
considered was the prior work undertaken by the Connecticut Department of Education 
in recommending approaches to developing standards of satisfactory progress, and adult 
education participants' past performance on the CAPP test. 

The following draft qualitative and quantitative standards are proposed for 
determining Job Connection clients' progress in education programs . There must be 
qualitative evidence of clients' progress in basic skills, ESL, academic, or high school 
completion programs as indicated by the standards that are listed below. There also 
must be quantitative evidence of clients' progress in education programs as indicated by 
the amount of time allotted to complete tlie course of study or program. 
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Basic Skills/ESL Programs . Clients' progress in basic 
skills or ESL programs must be assessed after 75 hours 
of instruction, provided that this amount of instruction 
occurs over a period of at least three months, or by the 
end of each semester (not to exceed four months) diat a 
client is enrolled in a program. This information can be 
one of the following: 

a. Progress in acquiring basic skills or English 
language skills as indicated by gain scores on a 
standardized instrument. The following are the 
minimal level of gains that clients are expected 
to achieve: 

- CAPP Survey Achievement Test or 
Employability Competency System (ECS) 
Test: a scale score gain of at least 3 points 
in reading or listening; or 

- TABE: a gain of at least 1 grade level (for 
programs not required to use CAPP); or 

- ABLE: a gain of at least 1 grade level (for 
programs not required to use CAPP). 

OR 

b. Progress in acquiring basic skills or English 
language skills as indicated by clients' mastery 
of a percent of competencies specified for the 
course. A list of the competencies, the percent 
that is required to be demonstrated (as indicated 
by the service provider), and the specific 
evidence of a client's demonstration of the 
competencies must be provided. 



High School Completion Programs . Clients' progress 
in a GED program, the External Diploma Program 
(EDP), or the Credit Diploma Program must be 
assessed at least by the end of each semester a client is 
enrolled in one of these programs. The standards for 
progress for these programs are as follows: 
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a. GED: The attainment of the GED certificate or 
an increase of at least points on one of the GED 
practice tests over the period of a semester; or 

b. External Diploma Program: For clients in the 
Generalized Assessment component of the EDP: 
the completion of at least two tasks with the 
demonstration of at least 50 percent of the 
competencies in each task; or for clients in 
Diagnosis: the movement from Diagnosis into 
Generalized Assessment as determined by the 
EDP; or for clients in the Individualized 
Assessment component of the EDP: the 
demonstration of the competencies required by 
the Individualized Assessment component of tiie 
EDP or the attainment of the high school 
diploma. This progress must occur over the 
period of a semester; or 

c. Credit Diploma Program: The attainment of 
progress over a semester as determined by the 
program's coordinator. For example, this may 
include the completion of a number of courses 
with a grade or C or higher, the attainment of 
credit for experiential learning or participation in 
community services, or the completion of other 
program requirements. 



■ Hi gher Education Programs . Clients' progress in a 
higher education program must be assessed at least by 
the end of each semester a client is enrolled. A client is 
expected to achieve the standard that is set by the 
institution in which he/she is enrolled for maintaining 
good academic standing. 

In addition to the qualitative and quantitative standards described above, the Family 
Support Act requires that information about JOBS clients' attendance be reported to the 
U.S. Department of Health and Human Services. This information is reported about a 
sample of clients' average attendance over a period of a month. In order for this 
information to be available to the Job Connection office, district DIM offices will need 
to collect information about clients' attendance in education programs on a monthly 
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basis. The proposed attendance standard is that clients must attend 75 percent of the 
classes over a month's period and may have no more than 3 unexcused absences in a 
semester. 

Another source of information that can be used to supplement the standards for 
education programs presented above is the professional judgement of service providers . 
Two types of judgements are possible: 1) judgements concerning clients' attitudes, such 
as service providers' assessment of clients' demonstration of a positive attitude or their 
understanding of the requirements for successful completion of the program; and 2) 
judgements concerning clients' behavior in class, such as service providers' assessment 
of the extent to which a client participates in class, works with other students, or 
demonstrates the ability to undertake long-range planning. It is proposed that standard 
forms be developed for service providers to use in documenting professional judgement 
and that this information be collected from service providers when a client's progress in 
a program is questionable or additional information is needed by the case manager to 
make a determination of the client's status. 

The following draft standards are proposed for determining Job Connection clients' 
progress in skill training program s, including classroom-based and on-the-job training 
programs. The standard for attendance described for education programs also applies to 
skill training programs. The standards for skill training programs are as follows: 



■ Classroom-Based Programs . Clients' progress in 
classroom-based skill training programs must be 
assessed at least by the end of each semester (not to 
exceed four months) that a client is enrolled in a 
program. Either entire programs or components 
(sometimes referred to as units) of a program may be 
used as the standard for instruction. Documentation of 
progress in a program or a component of a program can 
be one of the following: 

a. Progress as documented by a grade of C or 
higher for a course or a grade point average 
of C or higher. If a grade point average is 
used as the measure of a client's progress, the 
client may earn a D in a course as long as the 
grade point average is C or higher; or 
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b. Progress as indicated by clients' increase in 
score as measured by an instrument appro- 
priate for the skill being taught. The score 
gain that would be adequate given the type of 
instrument that is b^ing used must be 
designated; or 

c. Progress as indicated by clients' mastery of a 
percent of competencies specified for the 
course. A list of the competencies, the 
percent that is required to be demonstrated (as 
determined by the service provider), and the 
specific evidence of a client's demonstration 
of the competencies must be provided. 

■ On-the-Job Training Programs . Clients' progress in on- 
the-job training programs must be assessed at least by 
the end of each semester a client is participating in the 
program. The documentation may include a rating of 
the clients' attitude toward the work, behavior in 
carrying out job responsibilities, and other information 
about specific skills required for the job for which the 
client is being trained. A rating scale such as that 
proposed in the previous discussion concerning 
Professional Judgement could be used by service 
providers, along with a narrative statement of a client's 
progress. 



The following guidelines are proposed for the selection of quality service providers 
for skill training programs . 



■ Non-profit, vocational technical schools and private 
occupational schools must be approved by the 
Connecticut Department of Education; 

■ Service providers offering adult education classes must 
provide documentation that Job Connection clients 
enrolled in their adult education programs are making 
satisfactory progress as defined by DIM; 

■ A service provider must have secured employment for a 
minimum of 70 percent of Job Connection enroUees 
witiiin 6 months of their training completion date during 
1991-1992. This rate would increase or decrease with a 
change in the economic condition of the State; 
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■ A service provider whose students receive Federally- 
sponsored student loans must have a default rate of less 
than 20 percent (or a rate determined by the 
Connecticut Department of Education); and 

■ A service provider that is an institution of higher 
education must be approved by the State agency 
overseeing higher education programs. 

Determination of Probationary and Mitigating Circumstances . The Family Support 
Act includes considerations for a probationary period and provisions for mitigating 
circumstances in applying standards for determining clients' progress in education and 
skill training programs. The reconunendation for a probationary period is as follows: if 
a client does not meet the standard for progress specified for an education or skill 
training program, he/she may be placed on probation for a period of time to allow 
him/her the opportunity to meet the required standard of progress. It is also proposed 
that an institution's regulations regarding the placement of students on probation be used 
as the first source of information for determining a client's period of probation. If a 
program, agency, or institution does not have a stated probation policy, then it is 
proposed that a client be allowed to have a probationary period of one semester (not to 
exceed four months) to meet the required standard. 

There are several mitigating circumstances that may interfere with a client's ability 
to progress satisfactorily in a program or course of study. These include the following: 
a) clients' learning disabilities and learning difficulties, b) sickness, c) family emer- 
gencies, and d) homelessness. Case managers should gather information from the client 
and/or service provider about the type of circumstance and make a determination about 
the validity and severity of the difficulty. Once this determination is made, the case 
manager could extend the period of time in which the client is expected to meet the 
standai"d for satisfactory progress. The recommended period of time is a semester (not 
to exceed four months). 
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Implementation of Standards for Satisfactory Progress 

The success of the implementation of the standards for determining Job Connection 
clients' satisfactory progress in education and skill training programs and the guidelines 
for selecting quality programs will depend on three main factors: 

■ The extent to which State agencies (i.e., the Depart- 
ments of Income Maintenance, Education, Labor, and 
High Education) can work cooperatively in dissemi- 
nating information to service providers about the 
standards and in encouraging service providers to work 
with case managers in transferring information about 
Job Connection clients. It is recommended that the State 
agencies undertake the following: 

a. Disseminate information to service providers in each 
system about the standards and the importance of 
working with local Job Connection staff in providing 
client information and assistance; and 

b. Convene quarterly meetings with representatives from 
the State agencies during the first year of the imple- 
mentation of the standards to identify barriers to 
implementation and propose solutions to these barriers. 

■ The extent to which Job Connection loc;^J staff are 
trained and provided with support (i.e., standardized 
forms) to implement the standards and guidelines. It is 
recommended that case managers receive training in the 
following: 1) how to interpret the standards, 2) meth- 
ods for collecting information about client progress and 
mitigating circumstances from service providers and 
clients, 3) procedures for making decisions about client 
progress, and 4) procedures for collecting and inter- 
preting information about the quality of service 
providers; and 

■ The amount and quality of information that is available 
about programs and economic conditions that can be 
used by case managers in selecting service providers. It 
is recommended that DIM work with other State agen- 
cies in identifying sources of information about service 
providers and local economic conditions that could be 
disseminated to case managers on a periodic basis. 
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L INTRODUCTION 



The Federal Mandate 

As part of the education and training requirements specified in the Family Support 
Act of 1988, the Connecticut Department of Income Maintenance (DIM) is required to 
approve criteria or standards for determining the satisfactory progress of Aid to Families 
with Dependent Children (AFDC)/Job Connection clients enrolled in education and skill 
training programs. According to the legislation, these standards of progress must be: 

■ Based upon a consistent written policy; 

■ Measured on the basis of less than a year (such as a 
term or quarter); 

■ Include a qualitative measure of the participant's 
progress (such as a grade point average); 

■ Include a quantitative measure of the participant's 
progress (such as a time limit for the completion of the 
program or course of study); 

■ Developed by the educational institution or program in 
which the participant was enrolled; 

■ Approved by the State or local educational agency; and 

■ Approved by the State welfare agency. 

The standards also may include consideration for a probationary period and provision for 
mitigating circumstances (Federal Register, 1989). 

The Job Connection staff in DIM have responsibility for approving and imple- 
menting these standards. As specified in the Federal legislation, State welfare agency 
staff-such as tlie Job Connection staff-should undertake the following: 
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■ Require 18 or 19-year old custodial parents to accept a 
job or training if they fail to make good progress in 
educational activities; 

■ Make a participant's enrollment in a training or 
postsecondary activity contingent upon satisfactory 
piogress; 

« Use the definition of progress to determine if a 
participant has been appropriately placed in an 
educational or training activity; and 

■ Use the definition of progress to determine whether to 
continue to provide Job Connection special benefits to 
those enrolled in education and training (Talmadge, 
1990). 

Because of the complexity of the task of specifying standards of progress and the 
potential consequences for a participant who does not meet the standards. State welfare 
agencies were given a three-year period in which to develop and implement the 
standards. During this period, DIM has used the standards of progress that have been 
developed by service providers and those that have been negotiated with DIM-funded 
service providers. This experience provided the background information for DIM to use 
in specifying standards for determining Job Coanection clients' satisfactory progress in 
education and skill training programs. 

Activities in Support of the Mandate 

Since January 1990, the Job Connection staff in DIM have worked with the 
Connecticut Department of Education staff and others to examine the types of practices 
undertaken by education and skill training providers to assess clients' progress in 
programs. Issues regarding the types of reasonable standards that can be set for 
detemfiining progress and methods for transferring information between education and 
skill training providers and DIM district Job Connection staff have been discussed in 
numerous meetings involving DIM and Department of Education staff. These issues 
also have been the subjects of discussions among staff in local programs as they attempt 
to meet the requirements of the Federal legislation. 
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During this period, a number of State initiatives has been carried out that address the 
issues of client assessment and collection and interagency transfer of data. For example, 
the implementation of the Connecticut Adult Performance Program (CAPP) has provided 
adult education programs funded by the Department of Education with assessment 
instruments and procedures for placing clients in instructional programs and for 
measuring their progress. These data have been used by local Job Connection staff in 
assessing the extent to which clients are achieving their educational goals. 

One contribution from PROJECT LIFE (Learning Incentives for Employment) was 
the development of a client referral form that could be used by local education, income 
maintenance, and job traming agencies. The goal was to have a form that would 
facilitate clients' participation in services offered by these agencies. While the form 
never became part of a mandated data collection process, it did serve as an example for 
current efforts in refining the CAPP data collection system. 

Another initiative that has contributed to the development of client assessment and 
information exchange processes has been the CETO collaborative. This initiative has 
provided opportunities for local DIM district staff to work with occupational training, 
adult education, and higher education staff in serving Job Connection clients. These 
staff have experimented with written and oral methods for transferring information that 
are needed to determine clients' status in programs. 

A rich array of information concerning Job Connection participants' satisfactory 
progress in education and skill training programs has emerged from State-level staff 
discussions and the activities of the varied initiatives. Because of the breath and 
numerous sources of information, the DIM Job Connection administrative staff thought 
that it would be beneficial to have an independent consultant review this information and 
prepare draft standards of satisfactory progress. Therefore, DIM sought the assistance 
of COSMOS Corporation to undertake this task. 

COSMOS'S Project 

The Department of Income Maintenance, through an interagency agreement with the 
Connecticut Department of Education, commissioned COSMOS Corporation to carry out 
a project during the period January-July 1992, which had the following objectives: 
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■ Review existing information concerning the 
development of standards for satisfactory progress; 

■ Collect information from DIM district case managers 
concerning their procedures for determining satisfactory 
progress, their methods for collecting this information 
from service providers, and their methods for 
determining quality programs; 



■ Collect information from a small sample of States 
regarding their policies for determining satisfactory 
progress as specified in the Family Support Act of 
1988; 



■ Develop draft standards for determining satisfactory 
progress in basic skills, English as a second language 
(ESL), high school completion, higher education, and 
skill training programs; 

■ Develop draft guidelines for determining quality 
programs; and 



■ Identify issues that should be considered in the 

implementation of standards, particularly in terms of the 
types of staff training that might be provided. 



This document is the final report for the project and presents its findings. Described 
in Section II are the results of the review of existing information and the collection of 
new data concerning current practices undertaken by DIM district Job Connection staff 
in determining clients* satisfactory progress and selecting quality programs. Also 
discussed in this section is the information that was collected from five States regarding 
their policies for determining satisfactory progress. Described in Section III are options 
and recommendations for draft standards for determining satisfactory progress and 
guidelines for selecting quality programs. Finally, Section IV presents issues that should 
be considered in determining the final standards and their implementation procedures, 
particularly in terms of the need for staff training. 



5 

11. CURRENT PRACTICES IN DETERMINING SATISFACTORY PROGRESS 



Introduction 

This section of the report presents the findings from the collection of new and review 
of existing information concerning DIM district Job Connection staffs practices in 
determining the satisfactory performance of Job Connection clients in basic skills, 
English as a second language, high school completion, higher education, and skill 
training programs. Adaressed are the types of information collected by Job Connection 
staff, the methods they use to obtain this information from education and skill training 
service providers, and the factors that have facilitated and inhibited the transfer of 
information between local agencies. Also discussed are the criteria used by Job 
Connection staff to determine the quality of service providers to whom they refer clients 
and the barriers they have encountered in obtaining information to make these 
judgements. 

In addition to the information about Job Connection staffs' collection of satisfactory 
progress information, this section presents data that were collected from five State 
welfare agencies concerning their policies for determining satisfactory progress in 
education programs. Included in this discussion are the criteria for standards that have 
been discussed by State Job Connection and Department of Education staff in their 
meetings during the past two years. 

Data Sources 

A variety of existing sources of information was reviewed and new data were 
collected to obtain an understanding of the practices that currently are being used by 
local Job Connection staff to determine the extent to which clients' progress in programs 
is satisfactory and to select quality programs. The following information sources were 
examined as part of the project's activities: 

■ Notes from Regional Interagency Steering Committee 
(RISC) meetings held during 1990-1991; 
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■ Reports v/ritten about PROJECT LIFE; 

■ Memoranda issued by Job Connection and Department 
of Education staff; 

■ Policy documents issued by DIM and other State anu 
local welfare agencies; 

■ Sample data collection forms submitted by local Job 
Connection and local adult education program staff; and 

■ Data concerning CAPP assessment results during 1987- 
1990. 

In addition to analyzing existing information, the project's staff collected new 
information through the following four methods: 1) the conduct of focus groups with 
local Job Connection a^id adult education staff, and representatives from higher 
education and skill training agencies; 2) the conduct of a meeting with the project's State 
Advisory Committee; 3) the conduct of a mail survey of local Job Connection case 
managers; and 4) the conduct of telephone interviews with representatives from four 
State welfare agency Job Opportunities and Basic Skills (JOBS) staff. 

Focus Groups . Three separate focus groups were held in March 1992. These 
groups were comprised of representatives from the six DIM districts, and staff from the 
adult education and skill training programs who work with these districts. Each meeting 
was approximately two hours in length. During these meetings, participants were asked 
to describe their current practices for determining clients' satisfactory progress, the types 
of activities that they have undertaken in transferring information between agencies, and 
their concerns regarding the implementation of standards for determining satisfactory 
practice. Participants also were asked to send samples of data collection forms that they 
have used to communicate information about clients' progress. 

State Advisory Meeting . During January 1992, the project's State Advisory 
Committee was convened to discuss the design of the project and Committee members' 
experieiices to date discussing the development of satisfactory progress standards. The 
Committee is comprised of representatives from the Connecticut Department of Income 
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Maintenance, Department of Education (Bureau of Adult Education and Training), 
Department of Labor, and the higher education system. A member of the Adult 
Training and Development Network staff also is part of the Committee. During the 
meeting, the members discussed their main concerns regarding both the specification and 
implementation of satisfactory standards. 

Job Connection Case Manager Survey . During April 1992, a mail survey was sent 
to all local Job Connection case managers to collect information about the following four 
issues: 

- The types of education service providers that serve Job 
Connection clients; 

■ The practices that are used for determining the 
satisfactory progress of Job Connection clients in adult 
basic education, ESL, post- secondary, and skill 
training programs (classroom and on-the-job training); 

■ Factors that influence the determination of satisfactory 
progress; and 

■ Processes that are used to identify and select skill and 
job training programs for Job Connection clients. 

The survey was sent with a letter to each local Job Connection case manager, asking 
the case manager to return the survey directly to COSMOS Corporation. Respondents 
were told that their responses would be confidential, and the data would only be reported 
at the aggregate level by district. A letter describing the survey also was sent to the 
directors of the DIM district offices (and, in some cases, sub-offices). After the initial 
mailing of the survey, the project staff called the district office directors and asked them 
to encourage their case managers to return the surveys. Copies of the letters and survey 
are presented in Appendix A. 

A total of 88 surveys was sent to Job Connection case managers. Of these surveys, 
two were returned because the respondent was no longer at the address and 46 were 
returned completed (a response rate of 52 percent). Because the last survey was 
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received after data analysis had been completed, it is not included in the data tables 
presented later in this section. The response rates for each of the DIM districts were as 
follows: 1) Hartford-77 percent, 2) Norwich"75 percent, 3) Bridgeport-45 percent, 4) 
New Haven-33 percent, 5) Middletown~30 percent, and Waterbury-22 percent. 

State Welfare Agency Data Collection . An important component of the data 
collection for this project has been the information that was collected from other State 
welfare agencies concerning their policies regarding satisfactory progress standards. As 
an initial step in this process, the project's director interviewed the U.S. Department of 
Health and Human Services's Acting Chief of the JOBS Policy Branch to determine the 
availability of data regarding State practices in determining the satisfactory progress of 
JOBS clients in education and skill training programs. It was found that these data are 
not reported to the Federal JOBS Program office in the States' JOBS Plans or other 
reporting forms. Given this finding, the project staff identified States that were early 
implementers of the JOBS Pro gram on the assumption that these States would be more 
likely to have policies established for determining satisfactory progress. The project 
staff collected information from JOBS staff from the following five States: 1) 
California, 2) Michigan, 3) New Jersey, 4) Massachusetts, and 5) Pennsylvania. 

Telephone interviews were conducted with representatives .^rom each State except 
California, since COSMOS previously had collected the relevant information from this 
State. In addition to an initial telephone interview, several follow-up telephone calls 
were made to the JOBS staff in these States to clarify the information that was sent to 
COSMOS. 

Summary . The development of standards for satisfactory progress has been 
discussed extensively by State and local Job Connection, education, and skill training 
program staff in Connecticut. Through the convening of State-level meetings, the pub- 
lication of policy memoranda and reports, and the conduct of meetings among local 
providers. State policymakers and local program staff have had opportunities to suggest 
possible standards that might be considered and to identify the impediments to 
implementing these standards. 

Through the activities of this project, new information has been collected about the 



9 

current practices of local Job Connection staff in assessing their clients' progress in 
programs, and the factors that have assisted and impeded the staffs' collection of client 
information from other agencies. Data also have been provided about the factors that 
Job Connection staff consider in selecting local service provider referrals-when a choice 
is possible. Finally, policy information concerning standards of satisfactory progress has 
been collected from a sample of States that were early implementers of the JOBS 
Program. 

Data gathered from all sources were analyzed and integrated. The results of this 
analysis, which provide a background for the draft standards of satisfactory progress 
presented in this report, are discussed below. 

Information Used l o Determine Clients' Satisfactory Progress 

A critical element that should be considered in the establishment of standards of 
satisfactory progress is the type of information regarding standards that currently is 
available and used by local Job Connection staff in assessing their clients. Also 
important for the implementation of these standards is the ability of Job Connection staff 
to access information in a timeframe that meets Federal and State reporting require- 
ments. 

One objective of this project has been to determine the types of client information 
that local Job Connection staff have utilized in monitoring their clients' progress in 
education and skill training programs, as well as the processes they have employed in 
sharing client information across local agencies. While the survey of local Job 
Connection case managers was a main source of data about these topics, the information 
collected in the focus group meetings and in the review of RISC meeting notes and 
PROJECT LIFE reports also has contributed to our understanding of these topics. 

Range of Service Providers . A key responsibility of local Job Connection staff is 
their referral of clients to education and skill training service providers. In developing 
clients' education and employment plans, case managers must identify appropriate ser- 
vices for clients based on clients' educational need, geographical location, prior 
experience with a provider, as well as other factors. Case managers then often 
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collaborate with these providers in gathering ongoing information about clients' progress 
and attendance. 

According to the results of COSMOS'S survey. Job Connection case managers in the 
six DIM districts referred clients to 95 different education and job training agencies 
during the past year. Presented in Table 1 is a summary of the types and numbers of 
providers specified by Job Connection case managers who responded to COSMOS'S 
survey. Some agencies provide multiple types of services to clients in a district (e.g., 
occupational/vocational skills training and job search/vocational exploration), so that the 
numbers of agencies reported for each district may not represent separate agencies. 
Appendix B contains a listing of the agencies that have provided services to Job 
Connection clients during the past 12 months. 

Across all of the DIM districts, the greatest number of agencies utilized is those that 
provide occupational and skills training. Sixty-eight different agencies were listed as 
having offered these types of services. The number of adult education service providers 
is the second largest category of providers, with survey respondents reporting 46 
separate agencies. In addition, respondents listed 17 higher educational institutions and 
7 agencies that offer job search/vocational exploration. Survey respondents indicated 
only one agency as having provided on-the-job training. 

Types of Information . Local Job Connection staff have had several opportunities to 
report the types of information they use to determine their clients' progress in education 
and skill training programs. These opportunities have included a survey undertaken as 
part of PROJECT LIFE, discussions held during RISC meetings, and the focus groups 
and survey conducted as part of this project. The findings from these data collection 
activities are consistent and present a clear pattern of the types of data that both are 
accessible and utilized by case managers. This information is critical in determining the 
types of standards of satisfactory progress that will be feasible, since case managers' and 
service providers' prior experience will influence their capacity to implement any 
standards that are developed. 
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The results from the analysis of COSMOS'S survey concerning the types of 
information used by case managers are similar to the data gathered from other sources. 
Table 2 presents the findings from the survey. Case managers identified the following 
five categories of information: 1) measures of skill gain; 2) other education measures, 
including grade or grade point average, teachers' written and oral assessments, and 
program certificate; 3) attendance; 4) job skill attainment progress; and 5) measures of 
clients' behavior and attitude. As indicated in the table, the most frequently-used 
measure of skill gain is pass rate on the General Educational Development (GED), 
followed by the CAPP score. Sixty-nine percent of all respondents indicated that they 
use scores from CAPP testing to assess their clients' progress. Few respondents 
reported using information from the Test of Adult Basic Education (TABE) (13 percent) 
or Adult Basic Learning Examination (ABLE) (16 percent). These results reflect the 
practice of State-funded adult education programs of using the CAPP assessment system 
to measure student gain. 

In addition to measures of skill gain. Job Connection case managers consider other 
types of educational data in determining clients' progress. All survey respondents 
reported using a grade or grade point average (G.P.A.), and almost all (89 percent) said 
that they consider a program certificate as evidence of progress. In terms of teachers' 
professional judgement, the majority of the Job Connection case managers (80 percent) 
indicated that they collect teachers' written assessment of clients, while 60 percent noted 
that they rely on teachers' verbal assessment. These results show that the professional 
judgement of teachers has been an important data source for case managers in 
monitoring their chents' progress in programs. 

cosmos's survey also asked case managers to check whether they use attendance 
as one indicator of progress. All respondents reported using this information. 

In addition to collecting data about education measures and attendance, the survey 
inquired about case managers' use of information about chents' job skill attainment, 
behavior, and attitudes. Approximately one third of the case managers (36 percent) 
indicated that they collect information concerning clients' progress in programs that 
provide job skill training. While these programs were the largest number of referrals 
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listed by respondents on the survey, the least amount of information is collected from 
these programs concerning clients' progress. This may be due to a large variation in 
student testing and performance documentation practices by service providers. 

The final type of information gathered by case managers concerns clients' change in 
behavior and attitude. This information can address clients' progress in either education 
or job training programs. Almost half (49 percent) of the survey respondents noted that 
they consider change in a client's attitude as one indicator of progress, while one third 
(36 percent) noted that they accept data concerning clients' change in behavior. On the 
survey, case managers also were asked to indicate whether they collect information 
directly from clients concerning their progress. Almost half (49 percent) responded 
positively. 

Criteria for Determining Progress . While information about the types of information 
being used by Job Connection case managers in determining progress is useful for set- 
ting standards, also important are the criteria case managers have utilized in determining 
whether progress is satisfactory. On COSMOS'S survey, case managers were asked to 
describe the indicators of satisfactory progress that they have used for each of the 
information types discussed above. Presented in Table 3 are the criteria that were listed 
by survey respondents. As indicated in the table, few respondents chose to describe the 
criteria. However, the information that was provided is similar to that discussed during 
the focus group and RISC meetings. 

The same broad categories of criteria are applied by case managers across measures 
of education skill gain. As shown in Table 3, case managers generally use an increase 
in test score, class grade, or grade point average as an indicator of progress. While an 
increase in score or grade may be the criterion used by case managers, there is little data 
available about the amount of increase that is considered by case managers to be satis- 
factory. TTie information that is available also indicates that there is great variation in 
the score levels, grades, and attendance rates used by service providers to determine 
client progress. This variation is reflective of the current practice in DIM districts of 
having local service providers set their own standards for satisfaaory progress. 
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Table 3 



CRITERIA FOR DETERMINING SATISFACTORY PROGRESS 
SPECIFIED FOR EDUCATION AND TRAINING MEASURES 





Criteria 


Number of 


jvieasutc 


Responses 


Skill Gain Measures 








CAPP Score 


■ 


Above 223 


2 




■ 


Above 225 for college 


1 




mm 
■ 


Increase in score 


9 




■ 


Used for placement only 


5 




■ 


Increase in score 


1 




■ 


usee lor piacemenx oniy 


1 


ABLE Score 


■ 


Increase in score ^ 


2 




■ 


Used for placement only 


1 


Attainment of GED/ 


■ 


Increase in GED score 


4 


high school 


■ 


Complete requirement for certificate 


2 






vjrrade or or Detxer in uipioma program classes 


1 


Class Grade/Grade 


■ 


Grade or average of C or above 


6 




■ 


G.P.A. of 2.0 or above 


3 


(G,P.A) 


■ 


Standard specified by institution 


5 


Teacher's Written 


■ 


Statement of at least "good" or satisfactory progress 


3 


Assessment 


■ 


Statement of student's positive attitude toward 


3 






learning 




Program Certificate 


■ 


Completion of objectives 


2 




n 


/\lUlineCi level Ui Llallllllg 


2 


Other Measures 








Attpndanftp 


■ 


75% monthly attendance 


2 




■ 


xxegUiMi Aiieiiuaiiv^ ^uwi uwiiuv^^ 


3 




■ 


Attend 50% of classes 


1 


Job Skill Attainment 


■ 


Receipt of Certificate 


2 






XVIUVCIIICUL ilCAL ICVCl \Jl leaiillllg 


3 




■ 


Standard specified by provider 


2 




■ 


Attainment of job 


1 


Othpr M^ea5;urfts 








Change in Client's 


■ 


Initiates contact with support ser\'ices 


2 


Behavior 


■ 


Demonstrates ability to undertake long-range 


2 






planning 




Change in Client's 


■ 


Demonstration of positive attitude toward training 


8 


Attitude 


■ 


Increase in class participation/interaction with 


2 






instructor 




Information Reported by 


■ 


Articulation of problems and methods for solving 


5 


Client 




them 
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During the focus group discussions and in the RISC meetings, case managers and 
service providers discussed options for standards, such as an increase in score of three 
or four scale points on CAPP or an increase of one or two grade levels on other 
instruments. These data also point to the lack of agreement among service providers 
about specific criteria that should be used to determine client progress as well as the lack 
of empirical information from which to devise a standard. 

In considering clients' progress in institutions of higher education, case managers 
have indicated that they use a grade point average of 2.0 or higher for the semester in 
which a client is enrolled, or a class grade of C or better. This information was 
reported in the survey and in the focus group meetings. 

For measures other than tests or grades, case managers have used the attainment of 
a certificate or the completion of a program's requirements as indicators of progress. 
Case managers also look to the institution or service provider to set a standard and 
assess their clients using this information. 

There is also variation in the criterion used by case managers regarding attendance. 
While the Family Support Act's regulations specify that JOBS participants must have an 
average of 75 percent attendance during a month's time, case managers report using 
rates from 70 to 80 percent in making their assessment. 

In measuring progress in job skill training programs, case managers indicated in the 
survey that they consider the receipt of a certificate or movement to the next level of 
training as evidence. They also rely on service providers to specify a standard for 
assessing client progress. 

The final area in which information about criteria was collected concerns the 
measurement of change in clients' behavior and attitude. Case managers rely on their 
own observations and those of service providers in making these assessments, rather 
than on the collection of formal data through instruments or other systematic processes. 
As shown in Table 3, information such as clients' ability to plan and their initiation of 
contact with service providers are two examples of behavioral change. In the area of 
attitudinal change, the criterion that was cited most frequently by survey respondents 
was clients' demonstration of a positive attitude toward their training program. 
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Extent of Information Used . The majority of survey respondents use multiple 
measures to assess their clients' progress. Table 4 presents the numbers of measures 
case managers reported using for each information type. An analysis of the data in the 
table indicates that almost half (48 percent) of the respondents use three measures of 
skill gain, while 56 percent indicated that they use four "other education measures." In 
terms of behavioral and attitudinal measures, case managers are as likely to use two 
measures (36 percent) as they are to use no measures (36 percent). This information 
about the amount of data that case managers use in making their judgements about 
clients is important in determining whether the standards of satisfactory progress should 
include a variety of measures from which case managers can choose. 

In addition to inquiring about the types and amounts of information used by case 
managers, the survey also asked them to indicate whether there were other data that 
should be collected to determine clients' progress. Presented in Table 5 are the case 
managers' responses to this question by DIM district. Overall, respondents were equally 
divided about the need to collect additional data. However, an examination of the 
responses by district shows that the majority of respondents from Hartford and Norwich 
do not think that more data are needed, while those from the other districts would like 
more information. 

Summary . Job Connection case managers have established a broad network of 
service providers in referring their clients to education and skill training programs. 
These providers offer a range of services in basic education, ESL, high school 
completion, higher education, and skill training. Once case managers place clients in 
these programs, they collect multiple types of information to monitor clients' attendance 
and their progress in learning. While the case managers across the DIM districts use 
similar types of data, such as CAPP test scores, class grades, program certificates, and 
professional judgements about clients' attitudes, they do not necessarily assess clients' 
progress according to the same criteria. Rather, case managers have relied on their 
individual judgements or the standards that have been set by service providers in 
determining whether a client is advancing in a program. 
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An important component of the client data collection process undertaken by case 
managers is the methods they use to obtain information from service providers, A 
description of these methods and the factors that have facilitated and impeded the 
exchange of information between agencies are discussed next. 

Interagency Exchange of Information 

Job Connection case managers employ a variety of methods for gathering 
information about their clients' progress in education and skill training from service 
providers. With the implementation of the Family Support Act of 1988, DIM district 
offices have been responsible for monitoring their clients' participation in education and 
skill training programs in terms of attendance and some measure of program progress, 
such as grade level increase. Under a broad directive from the Department of Income 
Maintenance (e.g., see Ford, 1991), district staff have had flexibility in defining clients' 
progress and in establishing working relationships with agencies that provide services to 
Job Connection clients. While the district staff have had some success in developing 
procedures for transferring client information in a timeframe to meet the State Job 
Connection office's reporting schedule to the U. S. Department of Health and Human 
Services, they also have encountered difficulties in this process. 

Information Collection Processes . COSMOS'S survey of case managers identified the 
primary methods used by case managers to gather information from service providers. 
Table 6 presents the frequency of the methods that are used by DIM districts. The most 
common strategy is for case managers to obtain information through the mail, followed 
by case managers' oral communication with providers. 

An examination of the responses by districts indicates that all survey respondents 
from New Haven and Bridgeport reported that they use all methods except for placement 
testing. The respondents from the other four districts use these methods in varying 
degrees. 
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Impediments to Information Collection , Case managers have encountered a number 
of difficulties in collecting information about clients' progress from service providers. 
As shown in Table 7, these difficulties include information not being available in an 
interpretable form or in the required timeframe. Survey respondents indicated that their 
greatest difficulty was not being able to interpret the information provided to them 
concerning basic skills and other education measures. Two other factors that were 
identified by over half of the respondents were that attendance data are not provided in 
the required timeframe and that measures of job skill attainment are not available. 

This pattern of response was different for New Haven. As indicated in the table, 
survey respondents reported that the greatest impediment was not receiving attendance 
information and data about clients' performance on basis skills tests in a timely period. 

On the survey, case managers also were asked to describe the greatest difficulties 
they had encountered in attempting to collect information. Table 8 presents these 
results. In addition to problems with interpreting the information given to them, case 
managers noted that they do not understand how to define satisfactory progress and that 
they have difficulties matching clients with appropriate programs. While this second 
factor is not an aspect of transferring information, it is an issue that survey respondents 
identified as problematic in working with their clients. 

Facilitating Factors . When asked to rate the factors that have facilitated the 
interagency transfer of information, survey respondents indicated that the availability of 
information and their personal relafionships with providers were two key factors. Table 
9 presents these findings. While the geographical location of provider was not identified 
as a facilitating factor across all districts, it was cited as important in Nev. Haven, 
Norwich, and Middletown. 

Case managers also were asked to describe the factors that have helped them most in 
obtaining information from service providers. Table 10 presents these findings. The 
most-frequently cited factor in all DIM districts, except for Hartford, was case 
managers' relationship with service providers. The transfer of information between 
programs has been possible, in part, because of local agency staffs' willingness to work 
together in communicating about their clients. Their personal commitment to assisting 
clients has countered some of the difficulties staff have experienced with the 
information-transfer process. 
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Another factor that has facilitated the transfer of information is the availability of 
data. Almost half (44 percent) of the respondents said that being able to access 
attendance information in a timely manner was beneficial. 

The findings from the survey regarding the availability of attendance information and 
the interpretability of test data are equivocal. Respondents have cited these factors as 
being both problematic and helpful in monitoring their clients' progress in programs. 
One reason for this pattern of findings may be that there is variability among case 
managers within districts in their experiences with these processes. Thus, different case 
managers may have checked the factors as problematic and helpful. These results point 
to the need for further clarification about the role of these factors within DIM districts. 

Summary of Effective Practices for Transferring Information . The information 
collected from COSMOS'S survey, focus group meetings, and the review of RISC 
meeting reports indicates that Job Connection case managers and staff from service 
provider agencies are working together in a variety of ways to transfer client information 
across agencies. These methods include the use of referral forms that contain infor- 
mation about a client's demographic characteristics; educational and work history; 
performance on appraisal instruments; and the use of progress forms that contain data on 
clients' attendance and progress in basic skills, ESL, secondary education, or skill 
training programs. The progress forms sometimes include ratings of clients' behavior in 
class, such as clients' ability to receive constructive criticism, follow directions, and ask 
questions to clarify an assignment. In addition to the transfer of information in written 
form, case managers and service provider staff have established methods of commun- 
icating about clients through telephone discussions and meetings. These methods are 
particularly effective when a case manager has a question about the progress of a client 
and is able to discuss the issue with the service provider. 

Case managers generally rely on service providers to gather and transfer information 
about a client, but also receive information from clients about their progress. This 
practice is necessary in higher education programs, where attendance is not kept in a 
central office and the client is relied upon to provide the information to the case 
manager. 
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While case managers have been creative in working with staff from education and 
skill training agencies in sharing information, they also have experienced some barriers 
in gathering the data they need in an appropriate form and timeframe. Of particular 
concern have been case managers' difficulties interpreting information about clients' per- 
formance on basic skills measures, and the frequency with which attendance information 
is transferred between agencies. Also problematic has been the unavailability of 
information from skill training agencies regarding clients' progress. 

One factor that has facilitated communication between case managers and other 
agency staff is their personal relationships. The commitment of staff to work together to 
assist clients has mediated some of the structural difficulties staff have encountered in 
collecting the required information. 

Criteria for Determining Quality Programs 

One component of developing standards for determining clients' satisfactory progress 
in programs involves the specification of criteria for selecting these programs. A goal of 
COSMOS Corporation's project has been to identify the information that local Job 
Connection staff use in selecting service providers for their clients, as part of the process 
of specifying criteria for quality programs. Described next in the report are the types of 
information case managers consider when identifying potential service providers, as well 
as information that is not available but would be helpful in this process. 

Selection of Service Providers . In COSMOS'S survey. Job Connection case 
managers were asked to indicate the types of information that they use to identify and 
select agencies that provide education and skill training. While case managers do not 
always have a wide variety of agencies from which to choose, they do have some 
discretion in the referral process. 

Presented in Table 11 are the findings regarding the types of information 
respondents reported that they use to select service providers. Respondents rated the 
following four factors as most important in choosing service providers: 
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■ Type of skill training offered; 

■ Relationship between the skill training offered and the 
job market; 

■ Geographical location of the service provider; and 

■ Agency's rate of job placement. 

Also rated highly were case managers' personal experience with the service provider and 
the cost of the program. The case managers' choice of information reflects their 
concerns about being able to guide clients to skill training experiences that will result in 
a job placement. This concern was echoed in the focus group meetings, where case 
managers expressed their frustration with placing clients in skill training programs that 
are not matched to the local labor market or for which there are limited job 
opportunities. 

Information Needed for Selecting Programs . When asked to describe the types of 
information that they have not been able to access but that they would like to have, 
survey respondents noted that data about agencies' job placement rates were difficult to 
obtain as was information about program services and requirements. Table 12 presents 
these results. Since respondents were asked to generate information rather than react to 
a list of items, it is not clear that the other types of information listed on the table are 
available to case managers. For example, while information about default rates on 
student loans and dropout rates was identified by only 18 percent of the respondents as 
not being available, it is not reasonable to assume that, in fact, this information is 
available. Rather, it may be the case that case managers do not solicit this information 
about service providers. 

States' Criteria for Determining Satisfactory Progress 

Part of the data collection for this project has involved gathering information about 
standards for satisfactory progress from other States. Five States that were early 
implementers of the Federal JOBS Program-California, Michigan, New Jersey, 
Massachusetts, and Pennsylvania-were selected for study and telephone interviews were 
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conducted with representatives from each State except California. Policy documents 
were collected from the States and the information concerning the criteria for 
determining clients' progress in education programs was summarized. Table 13 presents 
these findings. While States also were asked to send information about criteria for 
determining clients' progress in skill training programs, limited written information was 
available. 

All of the States except for Pennsylvania provided specific information about the 
determination of clients' progress in programs. Pennsylvania's policy document 
addressed criteria for placing clients in programs, but did not specify the measures of 
progress. The information collected from States is presented in Table 13 according to 
education program type as defined by the State. California has the most clearly 
delineated standards, with the required scale scores on die Comprehensive Adult Student 
Assessment System (CASAS) Survey Achievement test being stated. Michigan uses 
competency tests that are administered by local providers and relies on these providers to 
set the standard. New Jersey does not specify the type of measure that should be used, 
but relies on individual service providers to judge clients' progress. Finally, 
Massachusetts specifies the instrument and levels for clients in ESL programs, and relies 
on local providers' use of competency tests for assessing clients' progress. 

The representatives from the States who provided this information indicated that 
these were working guidelines in many cases that were just beginning to be 
implemented. These individuals did not have information about the success of each State 
in carrying out these standards. 
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III. PROPOSED STANDARDS FOR DETERMINING CLIENTS* SATISFACTORY 
PROGRESS AND GUIDELINES FOR SELECTING OUALITY PROGRAMS 



Introduction 

The information collected in this project about Job Connection case managers' 
practices in gathering data and making decisions about their clients* participation and 
progress in education and skill training programs provides critical background 
knowledge for developing draft standards of satisfactory progress. As the results from 
the survey, focus group meetings, and review of RISC meeting reports indicate, case 
managers are employing a wide variety of methods to gather information and make 
decisions about their clients' progress. The data about staffs current practices offer an 
initial understanding of the types of standards that are feasible. However, the standards 
currently used by service providers have not been developed based on empirical data 
about the performance of clients and vary considerably across DIM districts. Because of 
this variation in development and practice, additional information must be utilized in 
specifying standards for determining clients' satisfactory progress. 

The policies that have been set by other State welfare agencies also provide 
comparative information that DIM can consider in approving standards. Since most 
States are in the process of setting standards, data are noi yet available concerning the 
success of the implementation of these standards. Thus, DIM*s work on standards 
should assist other States welfare agencies in their developmental activities. 

In addition to creating draft standards, one objective of this project has been to 
specify draft guidelines that can assist Job Connection case managers in selecting skill 
training service providers. The data collected about the information that case managers 
currently use in choosing skill training programs serve as a starting point in formulating 
guidelines for quality programs. 

This section of the report presents draft standards for determining clients* progress 
in education and skill training programs and draft guidelines for the selection of quality 
skill training programs. Discussed next are factors that should be considered in the 
establishment of these standards and guidelines. Also presented in this section are the 
mitigating circumstances that should be considered in applying the standards. 
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Framework for Standards and Guidelines 

The regulations for the Family Support Act of 1988 specify broad criteria for 
developing standards of satisfactory progress. These criteria, which were presented in 
Section I of diis report, were used by the project team as one guideline in writing draft 
standards. In particular, die following diree criteria were considered in die development 
of the standards: 

■ Clients' progress must be measured on the basis of less 
than a year (such as a term or quarter); 

■ The standards must include a qualitative measure of the 
clients' progress (such as a grade point average); and 

■ The standards must include a quantitative measure of 
clients' progress (such as a time limit for the completion 
of the program or course of study). 

In addition to these criteria, die project team identified diree odier factors diat 
should be considered in setting standards of satisfactory progress and quality program 
guidelines. These were: 1) die extent to which die standards and guidelines are 
reasonable and fair— i.e, are based on empirical data or on other information about 
clients' capacities to perform in a specific timeframe; 2) die feasibility of the standards 
and guidelines-i.e, the extent to which information needed to implement a standard is 
available and accessible to Job Connection case managers; and 3) the extent to which the 
standards and guidelines are flexible so diat diey can be modified to meet die special 
situations or needs of clients. 

Reasonable and Fairness Criterion . Since the implementation of standards for 
assessing the satisfactory performance o^ Job Connection clients has implications for 
their receipt of Job Connection Special Benefits and AFDC eligibility, it is important 
that these standards be reasonable and fair. One approach is to set standards that reflect 
the performance capacity of Job Connection clients. This involves developing a standard 
(e.g., the amount of increase in scale score that a client is expected to achieve in a time 
period, such as one semester of instruction) based on the known achievement levels of 
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clients or of adults with similar demographic characteristics, educational backgrounds, 
and life experiences. This type of standard is a relative one that is derived from actual 
rather than optimal or hypothetical client performance outcomes. A contrasting 
approach, where a standard is set based on a level that is optimal for clients, would 
utilize absolute rather than relative outcome levels. In specifying standards of progress 
for Job Connection clients, a reasonable and fair approach would be to set a relative 
standard based on empirical evidence. 

Another factor that must be considered in developing standards is the timeframe 
within which a client is expected to make satisfactory progress. Again, the most 
appropriate criterion to use in setting the standard for time would be a comparable group 
of adults* past experience. In the case of Job Connection clients, most basic skills 
service providers assess client progress using standardized instruments on a semester 
basis. One reason for this time period is that the instruments which are used by these 
service providers (e.g., CAPP test, TABE, ABLE) are not designed to be administered 
more frequently than after approximately 75 hours of instruction (assuming that this 
amount of instruction occurs in a timeframe of at least approximately three months). 
Thus, a reasonable timeframe for measuring clients* progress using standardized basic 
skills instruments appears to be after approximately 75 hours of instruction over a period 
of at least three months or a semester. 

Another component of the time issue concerns the amount of time instruction is 
available to Job Connection clients. For example, the length of time basic skills and 
skill training programs offer instruction varies considerably across the State. The 
standards for satisfactory progress that are set must take into account the amount of time 
that clients can access instructional programs. It would be unreasonable to set a 
criterion for progress that is based on an assumed amount of instructional time (e.g., 
progress as measured by a standardized test after 75 hours of instruction) that, in fact, is 
not available to clients. 

A final consideration regarding the development of reasonable and fair standards 
concerns the documentation of professional judgement. As was discussed in Section II 
of this report, case managers use their own judgements and those reported by service 
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providers in assessing the progress of their clients. In order to ensure that clients are 
being judged using similar criteria, guidelines for documenting professional judgement 
should be part of the standards of satisfactory progress. That is, case managers and 
service providers should address similar categories of clients' attitudes and behaviors 
when recording their judgements in these areas. 

Feasibility of Standards and Guidelines . The second factor that is important to 
consider in developing standards of progress and guidelines for quality programs is 
feasibility. The main issue regarding feasibility is the extent to which information for 
implementing standards and guidelines is available and accessible to case managers. For 
example, client attendance in programs is one type of standard of satisfactory 
performance. Education and skill training service providers document their clients' 
attendance according to different time periods. Any standard that is set regarding client 
attendance must take into consideration the extent to which this information is available 
to case managers. 

The availability of information also is an issue in developing guidelines for quality 
programs. Case managers have indicated that they would like to have data about 
agencies' job placement rates, institutions' student loan default rates, and the match 
between skill training programs and the local labor market. However, while tliis 
information may be available, it may not be accessible to case managers in a form and 
timeframe that is useful. The extent to which such information can be accessed must be 
considered in the development of guidelines for quality programs. 

Flexibility of Standards and Guidelines . A primary objective of the Federal JOBS 
Program is to assist welfare clients in achieving economic independence. The 
requirement that these clients participate in education and skill training programs was 
instituted to help clients improve their knowledge and skills and thus increase their 
likelihood of achieving economic independence. The use of standards for measuring 
clients' progress in education and skill training programs has two purposes: 1) to 
provide information that can be used to guide and assist clients as they participate in 
education and skill training programs, and 2) to provide data that can be used in 
monitoring the overall functioning of the Job Connection program and in determining 
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clients' receipt of special benefits. Because of these multiple purposes of standards and 
the potential impact on a client who is assessed as not making satisfactory progress, it is 
important that any standards that are set be flexible. The guiding principle is to set 
standards for Job Connection clients that are the same as those set for other clients 
participating in programs and to provide a process whereby case managers can consider 
Job Connection clients' special circumstances or needs in applying the standards. 

Draft Standards for Satisfactory Performance 

Background Data . The types of information that Job Connection case managers use 
to judge their clients' progress in education and skill training programs have been one 
source of data examined in developing standards. In addition, there are two other types 
of data that have been examined. The first is the prior work that was undertaken by the 
Connecticut Department of Education staff in recommending approaches to developing 
standards of satisfactory progress. The recommendation made by these staff (see 
Woolis, 1990) was to give service providers an array of client outcomes (with stated 
minimum levels of performance) from which to choose in assessing client progress. For 
example, the following categories of outcomes were suggested: 

■ Academic performance; 

■ Attendance, punctuality; 

■ Life and employability skills; 

■ Ability to set personal and career goals; and 

■ Self esteem. 

The approach that was envisioned was that local service providers would specify a range 
of possible outcomes for clients and use corresponding instruments and data collection 
processes to monitor their clients' progress. 
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The second source of data that has been examined is adult education participants' 
past performance on the CAPP test. In keeping with the assumption that a reasonable 
standard is one that is based on clients* actual rather than desired performance, the data 
analyses that COSMOS Corporation conducted for the Connection Department of 
Education during the years 1987-1988, 1988-1989, and 1989-1990 were summarized and 
are presented in Table 14. Shown in the table are the results for valid pre- and post-test 
gains on the CAPP Survey Achievement Test in reading and listening for each of three 
years for which data are available. The test data are from adult education program 
participants from across the State. While Job Connection clients were participants in 
these programs during the periods data were collected, they most likely represented a 
small percentage of the participants for whom data were collected. Thus, the gains in 
test score that are presented reflect the capacities of a student population broader than 
that of Job Connection clients. 

As indicated in Table 14, the amount of progress has differed by test level (which 
generally corresponds to class level) of participant. Another observation is that, for the 
most part, the average gain score has decreased as the number of participants has 
increased. Based on these data, it would be reasonable to expect that Job Connection 
clients could be expected to show an increase of at least three scale score points after 75 
hours of instruction provided over a time period of at least three months or one semester 
of instruction. While there are differences in growth between levels and between 
reading and listening, for the ease of implementation of the standard it is recommended 
that one score be used initially for the standards. The standards could be adjusted if 
data from the initial implementation results revealed that the differences between levels 
and reading and listening were significant. 

Draft Standards for Education Programs . The following draft qualitative and 
quantitative standards are proposed for determining Job Connection clients' progress in 
education programs . There must be qualitative evidence of clients' progress in basic 
skills, ESL, academic, or high school completion programs as indicated by the standards 
that are listed below. There also must be quantitative evidence of clients' progress in 
education programs as indicated by the amount of time allotted to complete the course of 
study or program. 
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■ Basic Skills/ESL Programs . Clients* progress in basic 
skills or ESL programs must be assessed after 75 hours 
of instruction, provided that this amount of instruction 
occurs over a period of at least three months, or by the 
end of each semester (not to exceed four months) that a 
client is enrolled in a program. This information can be 
one of the following: 

a. Progress in acquiring basic skills or English 
language skills as indicated by gain scores on a 
standardized instrument. The following are the 
minimal level of gains that cHents are expected 
to achieve: 

- CAPP Survey Achievement Test or 
Employability Competency System (ECS) 
Test: a scale score gain of at least 3 points 
in reading or Hstening; or 

- TABE: a gain of at least 1 grade level (for 
programs not required to use CAPP); or 

- ABLE: a gain of at least 1 grade level (for 
programs not required to use CAPP). 

OR 

b. Progress in acquiring basic skills or English 
language skills as indicated by clients' mastery 
of a percent of competencies specified for the 
course. A list of the competencies, the percent 
that is required to be demonstrated (as indicated 
by the service provider), and the specific 
evidence of a client's demonstration of the 
competencies must be provided. 

■ High School Completion Programs . Clients' progress 
in a GED program, the External Diploma Program 
(EDP), or the Credit Diploma Program must be 
assessed at least by the end of each semester a client is 
enrolled in one of these programs. The standards for 
progress for these programs are as follows: 
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a. GED: The attainment of the GED certificate 
or an increase of at least points on one of the 
GED practice tests over the period of a 
semester; or 

b. External Diploma Program: For clients in the 
Generalized Assessment component of the 
EDP: the completion of at least two tasks 
with the demonstration of at least 50 percent 
of the competencies in each task; or for 
clients in Diagnosis: the movement from 
Diagnosis into Generalized Assessment as 
determined by the EDP; or for clients in the 
Individualized Assessment component of the 
EDP: the demonstration of the competencies 
required by the Individualized Assessment 
component of the EDP or the attainment of 
the high school diploma. This progress must 
occur over the period of a semester; or 

c. Credit Diploma Program: The attainment of 
progress over a semester as determined by the 
program's coordinator. For example, this 
may include the completion of a number of 
courses with a grade or C or higher, the 
attainment of credit for experiential learning 
or participation in community services, or the 
completion of other program requirements. 



■ Higher Education Programs . Clients' progress in a 
higher education program must be assessed at least by 
the end of each semester a client is enrolled. A client is 
expected to achieve the standard that is set by the 
institution in which he/she is enrolled for maintaining 
good academic standing. 



Related Information for Education Standards . In addition to the qualitative and 
quantitative standards described above, the Family Support Act requires that information 
about JOBS clients' attendance be reported to the U.S. Department of Health and 
Human Services. This information is reported about a sample of clients' average 
attendance over a period of a month. In order for this information to be available to the 
Job Connection office, district DIM offices will need to collect information about clients' 
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attendance in education programs on a monthly basis. The proposed attendance standard 
is that clients must attend 75 percent of the classes over a month's period and may have 
no more than 3 unexcused absences in a semester. 

Another source of information that can be used to supplement the standards for 
education programs presented above is the professional judgement of service providers . 
Two types of judgements are possible: 1) judgements concerning clients' attitudes, such 
as service providers' assessment of clients' demonstration of a positive attitude or their 
understanding of the requirements for successful completion of the program; and 2) 
judgements concerning clients' behavior in class, such as service providers' assessment 
of the extent to which a client participates in class, works with other students, or 
demonstrates the ability to undertake long-range planning. It is proposed that standard 
forms be developed for service providers to use in documenting professional judgement 
and that this information be collected from service providers when a client's progress in 
a program is questionable or additional information is needed by the case manager to 
make a determination of the client's status. 

Draft Standards for Skill Training Proerams . The following draft standards are 
proposed for determining Job Connection clients' progress in skill training programs , 
including classroom-based and on-the-job training programs. The standard for 
attendance described for education programs also applies to skill training programs. The 
standards for skill training programs are as follows: 



Classroom-Based Programs . Clients' progress in 
classroom-based skill training programs must be 
assessed at least by the end of each semester (not to 
exceed four months) that a client is enrolled in a 
program. Either entire programs or components 
(sometimes referred to as units) of a program may be 
used as the standard for instruction. Documentation of 
progress in a program or a component of a program can 
be one of the following: 

a. Progress as documented by a grade of C or 
higher for a course or a grade point average 
of C or higher. If a grade point average is 
used as the measure of a client's progress, the 
client may earn a D in a course as long as the 
grade point average is C or higher; or 
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b. Progress as indicated by clients* increase in 
score as measured by an instrument 
appropriate for the skill being taught. The 
score gain that would be adequate given the 
type of instrument that is being used must be 
designated; or 

c. Progress as indicated by clients* mastery of a 
percent of competencies specified for the 
course. A list of the competencies, the 
percent that is required to be demonstrated (as 
determined by the service provider), and the 
specific evidence of a client's demonstration 
of the competencies must be provided. 

On-theJob Training Programs . Clients' progress in on- 
the-job training programs must be assessed at least by 
the end of each semester a client is participating in the 
program. The documentation may include a rating of 
the clients' attitude toward the work, behavior in 
carrying out job responsibilities, and other information 
about specific skills required for the job for which the 
client is being trained. A rating scale such as that 
proposed in the previous discussion concerning 
Professional Judgement could be used by service 
providers, along with a narrative statement of a client's 
progress. 



Draft Guidelines for Selecting Quality Programs 

The following guidelines are proposed for the selection of quality service providers 
for skill training programs. These guidelines were derived, in part, from the work 
undertaken by the City of New York Human Resources Administration (Day, 1990). 



Non-profit, vocational technical schools and private 
occupational schools must be approved by the 
Connecticut Department of Education; 

Service providers offering adult education classes must 
provide documentation that Job Connection clients 
enrolled in their adult education programs are making 
saUsfactory progress as defined by DIM; 
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■ A service provider must have secured employment for a 
minimum of 70 percent of Job Connection enrollees 
within 6 months of their training completion date during 
1991-1992. This rate would increase or decrease with a 
change in the economic condition of the State; 

■ A service provider whose students receive Federally- 
sponsored student loans must have a default rate of less 
than 20 percent (or a rate determined by the 
Connecticut Department of Education); and 

■ A service provider that is an institution of higher 
education must be approved by the State agency 
overseeing higher education programs. 

Determination of Probationary Period and Mitigating Circumstances 

The Family Support Act includes considerations for a probationary period and 
provisions for mitigating circumstances in applying standards for determining clients' 
progress in education and skill training programs. The recommendations for these 
considerations are described next. 

Probationary Period . If a client does not meet the standard for progress specified 
for an education or skill training program, he/she may be placed on probation for a 
period of time to allow him/her the opportunity to meet the required standard of 
progress. In keeping with the criterion of flexibility of standards described earlier in 
this report, it is proposed that an institution's regulations regarding the placement of 
students on probation be used as the first source of information for determining a client's 
period of probation. For example, the higher education institutions in the State and 
many of the agencies providing skill training programs have specific policies regarding 
probation. These may be used by the case manager in determining the flexibility that 
can be given to a client in meeting the standard. If a program, agency, or institution 
does not have a stated probation policy, then it is proposed that a client be allowed to 
have a probationary period of one semester (not to exceed four months) to meet the 
required standard. 

Mitigating Circumstances . There are several circumstances that may interfere with a 
client's ability to progress satisfactorily in a program or course of study. One barrier to 
client progress that has been discussed extensively by case managers and service 
providers is clients' learning disabilities and learning difficulties . This issue also has 
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been addressed in a recent report by Anderson (1992), and recommendations have been 
made for steps that service providers can take to identify clients with learning difficulties 
and to assist them in carrying out their educational programs. 

The likelihood that a Job Connection client with a diagnosed learning disability or a 
presumed learning difficulty will meet the standards of progress that are required for 
clients depends, in part, on the Job Connection Plan that has been set for the client. For 
example, if an adult enters the Job Connection program with documented evidence that 
he/she has a diagnosed learning disability, it is proposed that the case manager prepare a 
Job Connection Plan for this client that sets reasonable timelines for progress based on 
the type and extent of the learning disability. The case manager could develop the Plan 
and timelines in consultation with a knowledgeable service provider so that a plausible 
Plan and set of expectations are set for the client. 

If a client without a documented learning disability does not meet the standard of 
progress in an education program, the first step is for the case manager to determine the 
reason why the standard is not being met. This determination should be made in 
consultation with the service provider. As described in Anderson's paper (1992), a 
client may have a learning difficulty because of a physical impediment (e.g., a sight or 
hearing problem), a learning style difference, information processing difficulties, or 
other factors. In compiling the information about the client's lack of progress, the case 
manager should obtain an assessment of the client's work based on the service provider's 
professional judgement. This information could be used by the case manager to 
determine if the progress that had been made by the client was satisfactory given the 
client's mitigating circumstances (e.g., the learning difficulty that had been identified). 
If a client is determined to have a learning difficulty that prevents him/her for meeting 
the specified standard of satisfactory progress, the case manager could revise the client's 
Job Connection Plan to set outcomes that were reasonable based on the client's abilities. 

Other types of mitigating circumstances may prevent clients from progressing 
satisfactorily, such as sickness, family emergencies, and homelessness. Case managers 
should gather information from the client and/or service provider about the type 
circumstance and make a determination about the validity and severity of the difficulty. 
Once this determination is made, the case manager could extend the period of time in 
which the client is expected to meet the standard for satisfactory progress. The 
recommended period of time is a semester (not to exceed four months). 
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IV. IMPLEMENTATION OF STANDARDS FOR SATISFACTORY PROGRESS 



Introduction 

This section of the report discusses the issues that should be considered by Job 
Connection in planning for the implementation of the draft standards for satisfactory 
progress and guidelines for selecting quality programs. Described are the factors that 
must be addressed to ensure a successful implementation, particularly with regard to 
staff training. 

Factors Affecting Implementation 

The success of the implementation of the standards for determining Job Connection 
clients' satisfactory progress in education and skill training programs and the guidelines 
for selecting quality programs will depend on three main factors: 1) the extent to which 
State agencies (i.e., Departments of Income Maintenance, Education, Labor, and Higher 
Education) can work cooperatively in disseminating information to service providers 
about the standards and in encouraging service providers to work with case managers in 
transferring information about Job Connection clients; 2) the extent to which Job 
Connection local staff are trained and provided with support (i.e., standardized forms) to 
implement the standards and guidelines; and 3) the amount and quality of information 
that is available about programs and economic conditions that can be used by case 
managers in selecting service providers. Recommendations regarding each of these 
factors are discussed next. 

Interagency Cooperation . Critical to the success of this effort is the working 
relationship among the agencies who serve Job Connection clients. Through previous 
efforts and this project's Advisory Committee, representatives from the Departments of 
Income Maintenance, Education, and Labor and the higher education system have come 
together to discuss possible standards and the impediments to implementing any 
standards that are developed. The success of the implementation of the proposed draft 
standards will now depend on the ability of these agencies to carry out activities that can 
facilitate the implementation of the standards. The following recommendations are made 
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regarding the types of activities that can be conducted by State agencies (including 
higher education) serving Job Connection clients: 

■ Disseminate information to service providers in each 
system about the standards and the importance of 
working with local Job Connection staff in providing 
client information and assistance; and 

■ Convene quarterly meetings with representatives from 
the State agencies during the first year of the 
implementation of the standards to identify barriers to 
implementation and propose solutions to these barriers. 

Job Connection staff may also consider disseminating information about the standards 
directly to service providers not funded by these State agencies, such as proprietary 
schools. 

Another aspect of interagency cooperation involves the development of forms that 
can be used across local programs to share information. For example, the current 
revision of the CAPP management information system forms provides an opportunity for 
the Department of Education and DIM to review the form to ensure that it can facilitate 
the transfer of information, such as attendance and CAPP scores, from education service 
providers to case managers. It is recommended that representatives from the two 
agencies review the forms to maximize the use of the forms in interagency transfer of 
information. 

Staff Training . The most important component of the implementation is the training 
and follow-up support that will be provided to case managers. Case managers will need 
to receive training in the following: 1) how to interpret the standards, 2) methods for 
collecting information about client progress and mitigating circumstances from service 
providers and clients, 3) procedures for making decisions about client progress, and 4) 
procedures for collecting and interpreting information about the quality of service 
providers. 

It is recommended that an initial training of four hours be conducted to review these 
topics. Two or three representatives from each of the DIM districts should be required 
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to participate in the training, with the understanding that one individual from each 
district will serve as the key person for assisting other case managers in implementing 
the standards. The first part of the training should be the discussion of the standards. 
The standards should be presented to the training session participants in a brief 
document not to exceed five pages. The next part of the training session should be 
devoted to a discussion of the practices case managers currently are using to collect 
information from service providers and how these practices can be expanded. Brief 
descriptions of effective practices could be presented by case managers. Also discussed 
should be the circumstances under which clients can be expected to collect and deliver to 
the case manager information about his/her progress in a program. The third topic 
discussed in the training should be the interpretation of data. Simulated client records 
could be used in an interactive exercise to provide case managers with experience in 
reviewing client information and making a decision about a client^s progress in a 
program. The last topic covered in the training should be the guidelines for selecting 
quality programs, including a discussion about the types of information that currently are 
available to case managers and resources in the community that might be consulted. 

After the initial training, follow-up technical assistance may need to be provided to 
case managers by Job Connection staff. The feasibility of this process and the types of 
information that would be given through telephone technical assistance are issues that 
should be addressed by Job Connection staff. 

Provision of Information . Case managers currently use a variety of information to 
assess the quality and characteristics of service providers to whom they refer clients. 
The results of COSMOSES survey indicate that more and better information is needed to 
assist case managers in this process. It is recommended that DIM work with other State 
agencies in identifying sources of information about service providers and local 
economic conditions that could be disseminated to case managers on a periodic basis. 
For example, the Department of Education could provide a list annually of the State- 
funded basic skills programs and the hours of their operation. This would assist case 
managers in selecting an appropriate service provider for their clients. 
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Summary . The implementation of the standards for satisfactory progress will 
depend, in large measure, on the willingness of State and local agency staff to work 
together in collecting and disseminating information that can facilitate Job Connection 
clients' education and training. This joint work can be facilitated by developing forms 
for transferring information and by supporting local staff through training and technical 
assistance activities. The practices currently underway in the DIM districts are starting 
points for enhancing the system of services for Job Connection clients and for improving 
their capacities to attain economic independence. 
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A-1 



May 8, 1992 



Dear Director: 

As you are aware, the Department of Income Maintenance 
(DIM) , in collaboration with the State Department of Education, 
has procured the services of COSMOS Corporation to develop 
guidelines for implementing a consistent standard of progress for 
Aid to Families with Dependent Children recipients who partici- 
pate in the Job Connection. As part of the development of these 
guidelines, COSMOS is conducting a survey of DIM staff who work 
with Job Connection clients. The survey is designed to collect 
the following information: a) the current practices of DIM staff 
in determining the satisfactory progress of Job Connection 
clients, b) the factors that have facilitated and impeded DIM 
staffs' work with education and job training service providers, 
and c) the criteria that DIM staff use in determining the quality 
of skill and job training programs. 

During March 1992, focus groups were held with Job Connec- 
tion staff and representatives from education and job training 
service providers that work with Job Connection clients. The 
focus group participants identified processes they use to share 
information about the progress of these clients as well as diffi- 
culties they have encountered in obtaining information and 
identifying appropriate services for clients. This survey is 
intended to collect additional information about these issues 
that will help in the development of guidelines for determining 
Job Connection clients' satisfactory progress. 

Each Job Connection worker in your district is being sent a 
copy of the survey and is requested to complete and return the 
survey to me by May 24, 1992. The information provided by the 
respondents on the survey will be confidential, and the results 
of the survey will only be reported in aggregate form. A stamped 
return envelope has been included with the survey. Enclosed is a 
copy of the survey for your information. 



ERIC 



89 



Thank you for your assistance in the survey. The informa- 
tion provided by your staff is an important component of the work 
that we are undertaking for DIM. Please do not hesitate to call 
me at (202) 728-3939 if you have any questions about the survey. 

Sincerely, 



Judith A. Alamprese 
Director, Education & 
Training Group 



Enclosure 



May 8, 1992 



Dear Job Connection Staff Member: 

As you are aware, the Department of Income Maintenance 
(DIM) , in collaboration with the State Department of Education, 
has procured the services of COSMOS Corporation to develop 
guidelines for implementing a consistent standard of progress for 
Aid to Families with Dependent Children recipients who partici-- 
pate in the Job Connection. As part of the development of these 
guidelines, COSMOS is conducting a survey of DIM staff who work 
with Job Connection clients. The survey is designed to collect 
the following information: a) the current practices of DIM staff 
in determining the satisfactory progress of Job Connection 
clients, b) the factors that have facilitated and impeded DIM 
staffs' work with education and job training service providers, 
and c) the criteria that DIM staff use in determining the quality 
of skill and job training programs. 

During March 1992, focus groups were held with Job Connec- 
tion staff and representatives from education and job training 
service providers that work with Job Connection clients ♦ The 
focus group participants identified processes they use to share 
information about the progress of these clients as well as diffi- 
culties they have encountered in obtaining information and 
identifying appropriate services for clients* This survey is 
intended to collect additional information about these issues 
that will help in the development of guidelines for determining 
Job Connection clients' satisfactory progress. 

Enclosed is a copy of the survey for you to complete. 
Please return the survey to me at COSMOS Corporation in the 
enclosed envelope by May 24, 1992. The information that you 
provide on the survey will be confidential, and the results of 
the survey will only be reported in aggregate form. 

Thank you for your assistance in the survey. The informa- 
tion that you provide is an important component of the work that 
we are undertaking for DIM. Please do not hesitate to call me at 
(202) 728-3939 if you have any questions about the survey. 

Sincerely, 



Judith A. Alamprese 
Director, Education & 
Training Group 
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Name: District: 

(optional) Office: 
Position: 



SURVEY OF DEPARTMENT OF INCOME MAINTENANCE 
DISTRICT OFFICES REGARDING PRACTICES 
FOR DETERMINING SATISFACTORY PROGRESS 
OF JOB CONNECTION CLIENTS 



This survey is intended to collect information about the following four issues: 1) the types 
of education service providers that serve Job Connection clients; 2) the practices that are used for 
determining the satisfactory progress of Job Connection clients in adult basic education, English-as- 
a-second-language, post secondary, on-the-job training, and skill training programs; 3) factors that 
influence the determination of satisfactory progress; and 4) processes that are used to identify and 
select skill and job training programs for Job Connection clients. 



Please list (in order of importance) up to 10 education and job training agencies to which you 
have referred Job Connection clients most frequently during the past 12 months. List the 
agency and the type(s) of programs or services utilized (e.g., basic skills instruction, 
vocational training). 

Agency Type of Program/Service 



9Z 
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2. Please check the types of information that you use to determine a Job Connection client's 
satisfactory progress in an education or training program. Also, please describe the level of 
performance or indicators of behavioral or attitudinal change that you use in determining 
whether a client's progress is satisfactory (e.g., specific gain in score; rate of attendance; 
types of behavioral indicators). 

Information Type Indicator of Satisfactory Prog ress 

Check if Measures of 

Used Skill Gain 

CAPP Score 

TABE Score 

ABLE Score 

Attainment of GED/high school diploma 

Other (Please specify) 

Other Education Measures 

Class grade/Grade point average 

Teacher's written assessment 

Teacher's verbal assessment 

Program certificate 

Other Measures 

Attendance 

Measures of job skill 

attainment (please specify) 



Indicators of client's change in 
behavior (e.g., interaction with 
family and children, job-keeping 
behaviors — please specify) 

Indicators of client's change in 
attitude (e.g., attitudes about educa- 
tion and work — please specify) 

Information reported by client 
(please specify) 



Other (please specify) 
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3. In addition to the types of information listed in question 2, are there other data that you 

would like to collect about a client's satisfactory progress? Yes No 

If yes, please describe. 



4. How do you obtain the information listed in Question 2 about a client's progress in an 
education or training program? Check all that apply: 

Administer assessment/test instruments myself 

Obtain written information from education and training providers by mail 

Obtain written information from education and training providers through discussions 

and/or meetings 

Obtain verbal information from education and training providers 

Obtain verbal information from client 

Other (please specify) 



5A. Please check the difficulties that you have encountered in attempting to obtain information 
from service providers about clients' satisfactory progress in Job Connection. Check all that 
apply. 

Basic skills test results/information about other education measures are not 

available in a form that can be interpreted 

Basic skills test results/information about other education measures are not 

available in the required time frame 

Attendance information is not available in the required time frame 

Measures of job skill attainment are not available in a form that can be 

interpreted 

Other (please describe) 
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5B. Please describe the two difficulties that have been the most problematic and v/hy. 



6A, Please check the factors that have facilitated the collection of information from service 
providers about clients' satisfactory progress. Check all that apply. 

The availability of test, attendance, and/or other information in a written form 

The geographical location of the service provider 

Your personal relationship with staff at the service provider 

Other (please specify) 



6B. Please describe the two factors that have been the most helpful and why. 



Please check the types of information you use to identify and select education, skill, and job 
training programs for Job Connection clients. Check all that apply. 

Rate of job placement Default rates for student loans 

Types of skill training offered by Client's preference of service 

program 

Geographical location Drop out rates 

Personal experience/prior knowledge of Availability of support services 

provider 

Information from other clients who Relationship between area of skill 

attended the program training and labor market 

Length of program Cost of program 

Recruitment by provider Other (please specify) 



Please describe the difficulties that you have encountered in determining the quality of 
education, skill, and/or job training programs. 



In order to help you in referring clients to programs, what information about the quality of 
education, skill, and job training programs would you like to have access to, that currently is 
not available? 



Thank you for your cooperation in completing tiiss sinvey. 
Please return the survey in the enclosed envelope to: 



Judith Alamprese 

COSMOS Corporation 

1735 Eye Street, N.W., Suite 613 

Washington, D.C. 20006 
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LIST OF DIM DISTRICT EDUCATION AND 
JOB TRAINING AGENCY PROVIDERS 
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